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Independent Test Capability 

• Introduction to Software-Only-Simulation 

• Process and approach for simulation and hardware 
modeling 

• Independent Test Capability (ITC) 

• Jon McBride Software Testing & Research Lab (JSTARJ 

• Infrastructure, Deployment, and Users 

• Technologies Developed 

• Development Evolution of Spacecraft 
Simulators 

• Closing Remarks 

• Lessons Learned 
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Software-Only-Simulation 

Introduction 
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Software-Only-Simulation 


Introduction 


Independent Test Capability 


• Software-Only-Simulation is a complete software 
representation of modeled hardware components and 
software emulators 

• Together, the components form a complete spacecraft 
simulator 

• Software-Only-Simulator provides complete control of CPU, 
Time, and Memory 

— Can stop all execution for debugging. 

— Can peek/poke memory, perform fault injection 

• Spacecraft simulator used for: 

— Independent Testing (IVV) 

— Operator Training 

— Augment Project Hardware Testing 


NASA IV&V Independent Test Capability 


4 


Software-Only-Simulation 

Introduction 


@1TC 

Independent Test Capability 


Simulator Components 


Modeled 

Spacecraft 

Hardware 

cPCI, Spacewire, 1553, 
Ka-band, S-band, FPGAs, 


Instruction Set 

Simulator 

Simics/Qemu 

ppc750, ppc401, LEON3, etc 

Unmodified Flight 
Software Binary 


o 


/\ 

Independent 

Testing 


Halt Entire System 

Memory Analysis 
Fault Injection 


i 


Operational 
Ground System 




Spacecraft 

Scenarios 


What-IF Scenarios 
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Software- Only-Simulation 
Introduction 

Simulator Development Process 




Test System 
Project Plan 
T2501 



• Kickoff Meeting 

• Test Readiness 
Review 

• Release Readiness 
Review 

• Customer Feedback 
/ Post-Mortem 


Test System 
Roadmap 


T2502 



Test System 


Requirements 
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NASA IV&V Independent 
Test Capability (ITC) 
Introduction 
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©1TC Independent Test Capability (ITC) 

Introduction 


Independent Test Capability 


Charter 

Acquire, develop, and manage adaptable 
test environments that enable the dynamic 
analysis of software behaviors for multiple 
NASA missions 


Dynamic Analysis is performed on flight 
software to verify software behavior 
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#ITC Independent Test Capability (ITC) 

Independent Test Capability Introduction 


ITC Develops System 
Simulators 

— Experts in Hardware 
Modeling and Distributed 
Simulation 

— Experts in Simulator & 
Software Integration 



NASA IV&V Independent Test Capability 


9 



@1TC 

Independent Test Capability 


Jon McBride Software Testing & 
Research (JSTAR) Laboratory 


• Cloud-based infrastructure 
using server and desktop 
virtualization 


• Large scale simulator 
deployments 

• Hardware-in-the-loop and 
software-only test 
environments 



Integration of COTS and 
GOTS software tools to 
support V&V activities 
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Jon McBride Software Testing & 
Research (JSTAR) Laboratory 


Virtualized Deployment 


ag GO-SIM3.5 Running 


vApp Diagram 


Virtual Machines 


Networking 


S. O O O O 9r 
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ITC Technologies 
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NASA Operational Simulator (NOS) 


Software-only simulation 
architecture 

Capable of executing unmodified 
flight software 

Custom layered-architecture 
middleware 

Dynamic interception capability 

Reusable software modules and 
scripts 

Virtual machine deployment 


■ Virtual Computing Platform 


^ Hardware I Processor I Flight 

Models I Model(s) I Software 


Test Driver 
(Ground System) 



Typical NOS Architecture 
(Space Domain) 
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Plug-and-Play Hardware Models 




Processors, 

Boards, 

Racks 


Use of Operational Ground 
Systems Software 



Instrument Model Framework 

Instrumentl 

Subaddress HandlerA -> FunctionA 
Subaddress HandlerB -> FunctionB 

Subaddress HandlerN -> FunctionN 

InstrumentX 

Subaddress HandlerA FunctionA 
Subaddress HandlerB -> FunctionB 

Subaddress HandlerN -> FunctionN 


Internal Bus Monitoring 




1 MAC B 


5 Solar Array Drive B 
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11-' DC A 
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NOS Middleware 


Specialized Layers 




Base Layer Communications 


Deployment & Maintenance 



Virtualization 
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Independent Test Capability 


Overview 


S Offers re-usable communication 
mechanism 

• Ensures consistent and 
correct data passing 

S Provides synchronization 
between distributed 
applications 

S Flexible and extensible design 

• Can be extended to 
incorporate any 
communication protocol 


Features 


S Transport agnostic 
S Cross platform C++ 
implementation 
S Robust User API 
S Specialized User API Layers 

• MIL-STD-1553B 

• ESA SpaceWire 

• Discrete Signals 

• Time Synchronization 

S Interception allows for V&V 
analysis 

• No modification to software- 
under-test 
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System Under Test 


MIL-STD-1553 

SpaceWire 

Discrete 

Time Additional 

_ , Protocols as 

Synchronization Needed 


NOS Core Middleware with 
Interception Capability 

System 

Monitoring 

Bus 

Analyzer 

I/O Interface Layer 
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Virtual Oscilloscope 

- Virtual CompactPCI (cPCI) Analysis 

- Board-Level Signal Analysis 

Virtual MIL-STD-1553 Bus 


Gxl6d23 
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0xl6d04 
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- Bus Controller with XML Defined Schedules 


- Remote Terminal 

- Bus Monitor/Logger 

- PASS3200 Software Emulator 

Virtual SpaceWire Router 


<?xml version="l . 0" encodings 11 utf- 8 "'?> 

<BusContrQl le r_S chedule 

xjr.lns : x s i= 11 http : / /ww . w3 . org/2 001 /XiMLS cheoa- ins tance " 
xmlns="http : / /itc . iw.nasa , gov/1 . G/bc_s chedule" 

3 x 3 i : s chemaLo c at i on= 11 http : / / i tc . iw , nas a . gov/ 1 . 0 /bc_s chedi 

<1 — This XML file serves as a basic example of a EC schedule fo: 
3 <BCSchedule Name= ll bc_schedule_exaiiiiple 11 Repeat=" true" DoubleEi 
3 < Frame FirstMsgDelaYMicro5ec="10"> 

<Message MessageNum="l" RT="30" SA="14" Wo rdCount= " 1 
^Message RT="29" SA="l n WordCount= M 4 11 GapTimeMicreSe< 
^Message RT="30" SA="14" WoEdGount^ 1 ' 1 11 GapTimeMicrciSi 
^Message RT="'29" SA="14" WordCQunt=" , l 1 ' GapTimeMicroS* 
</ Frame > 


NASA IV&V Independent Test Capability 


17 


NOS Dynamic Interception 


0ITC 

Independent Test Capability 


Node A # 



Block 
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Evolution of ITC 
Spacecraft Simulators 
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Evolution of ITC Spacecraft 

Simulators 



Global Precipitation Measurement (GPM) 
Operational Simulator (GO-SIM) 

Closed-loop simulator including unmodified 
operational ground system, unmodified flight 
software, environmental simulator, and science 
instrument simulators 



James Webb Space Telescope (JWST) 
Integrated Simulation and Test (JIST) 

Simulator that demonstrates reusable NOS 
technologies can be applied to other NASA 
missions 



Deep Space Climate Observatory 
(DSCOVR) 

Turn-key modeling effort for spacecraft C&DH 
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Evolution of ITC Spacecraft 

Simulators 


GPM Operational Simulator 

(GO-SIM) 
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GPM Operational Simulator 

GO-SIM 


Components 

1 Capabilities 

• COTS Emulator 

• Primary Instrument 
Simulations (GMI/DPR) 

• GPM Ground System 

• GSFC Goddard Dynamic 
Simulator (GDS) 

• NOS Middleware 

• GPM Hardware Models ^ 

• Load and run unmodified 
flight software binaries 

• Execute test flight scripts 

• Single-step debugging 

• Inject errors via ground 
system and NOS middleware 

• Stress system under test 


NASA IV&V Independent Test Capability 


22 
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Independent Test Capability 



NASA IV&V Independent Test Capability 


23 


@1TC 

Independent Test Capability 


Evolution of ITC Spacecraft 

Simulators 


James Webb Space Telescope (JWST) 
Integrated Simulation and Test (JIST) 
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JWST Integrated Simulation and 


Test (JIST) 


Independent Test Capability 


• Software-only spacecraft simulator 

• Flexible environment to support V&V activities 

• Unmodified ground system and scripts 

• Unmodified software-under-test binaries 

• Integration of COTS, GOTS and in-house developed 
components 

• Custom hardware models 

• Automated Testing Framework 

• Fault Based Testing 
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JIST Architecture 
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JIST Architecture 
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Evolution of ITC Spacecraft 

Simulators 


Deep Space Climate Observatory (DSCOVR) 
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DSCOVR Architecture 
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Mission Training Simulator (MTS) 
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Simulator Level-of-Effort 
Comparison 


Year Usage 

Simulator 

Effort 

Prototype 

(Basic 

C&DH) 

Complexity 

Users 

2011-2014 

GO-SIM 

2 FTEs 

6 Months 

Medium 

IV&V, 

GPM Project Testers 
Launch Support 

2012- 

Ongoing 

J 1ST 

2 FTEs 

4 Months 

Very High 

IV&V, 

JWST Test Labs, 
JWST Operations 

2013- 

Ongoing 

DSCOVR 

1 FTE 

2 Months 

Low 

DSCOVR Testers 
DSCOVR Operations 
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• Establishment of a reusable simulation architecture has 
proven to save costs and reduce future effort 

• Automate tests and deployments as much as possible as it 
allows for engineers to focus on more challenging tasks 

• Hardware modeling should focus on the minimum needed 
in order for the flight software to execute. Establish this 
baseline then augment to support full V&V dynamic testing 
using an iterative process. 

• Spend considerable time writing unit tests for the 
hardware models. When things go wrong, debugging is 
very difficult. 

• Integration of simulators to form a system will require 

significant development labor, cost, and time. 
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• Web Page 

- http://www.nasa.gov/centers/iw/jstar/TSTAR.html 

• E-Mail 

- Justin.McCarty@TMCTechnologies.com 

- Scott.Zemerick@TMCTechnologies.com 

- Justin.R.Morris@nasa.gov 

• Contact us for... 

- Demonstrations of test beds 

- Middleware usage agreements 

- Simulator development 

- Hardware modeling 

- V&V Services, HWIL Testing, Performance Testing 
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